Welcome to the guidance documentation for using Pavlovia to create and run surveys online.
There is a table of contents on the left to help you navigate this guide and its main sections. Clicking on a section in the table will take you to the section and display subheadings within this section.
Please be aware this is a live document and is subject to change as I expand the guidance.
If you wish to use Pavlovia to create a survey or put an experiment online, please contact me at lcamus@qmu.ac.uk if you haven’t already to get set up.
I will create an account for you and send you the link for resetting your password. You will then be able to login and get started.
Please note that unlike Qualtrics, I do not have control over your login credentials, and will not be able to help with issues regarding forgotten login credentials. Should you forget your password, the Sign in page has a “Forgot your password?” link for you to set a new password.
When you first log in, there is a chance you will land on the GitLab end of Pavlovia, which is not where you will be able to create and manage your surveys and online experiments. Check the website link, if it starts with https://gitlab.pavlovia.org/ you are not in the right place. Instead, make sure you go to https://pavlovia.org/dashboard.
Pavlovia’s survey builder enables you to create surveys similar to those you may have made in Qualtrics. This page will provide guidance on how to get started with Pavlovia. If you have any questions, please contact me at lcamus@qmu.ac.uk.
Each section below covers different aspects of Pavlovia surveys - it is recommended you read all of them in detail for best results.
Below are the latest updates to Pavlovia Surveys at the time of writing (06 February 2025).
https://run.pavlovia.org/pavlovia/survey-2024.3.0/?surveyId=08016aae-5160-42e2-869f-7f5402773d93
There is now the option to save participant responses on each page
(requires using version 2024.3.0). To enable this, navigate to the
Overview tab in your survey on Pavlovia, then find the Settings section
and toggle the “Save on each page” option. This happens in real time,
which means a participant’s response is saved immediately upon moving on
to the next page of a survey.
As a result, if you download your survey results at various times during
data collection, you might get some participants’ responses at different
stages of completion. If you are going to use this function, it is
recommended to choose a date and time for a final download of your
survey responses, and to not work on your data before then.
Once you are signed into Pavlovia, you will land on your Dashboard, where you can see all your account’s details. For the purposes of creating a survey, we are only interested in the Surveys tab (see picture below, in the red box). Click on the Surveys tab to start.
Step 1
Once in the Surveys tab, you will see options to create or import surveys. Currently the import option is still rudimentary, so we will create surveys from scratch. Click on the New Survey button as shown in the picture below.
Step 2
A new survey will appear in the list of surveys (see picture below for example).
Step 3
Before proceeding to the design of the survey, you will need to rename it to something more descriptive/clearer, as shown in box 1 in the picture below. To do so, click on the survey’s current name, delete the existing text, and rename your survey.
Once this is done, click the “Open” button to open the survey in a new window to start designing it (box 2 below).
Steps 4 and 5
Once you open your new survey, you will be directed to the survey designer as pictured below.
Survey designer
The Designer tab provides you with an interface to allow you to click or drag options for your survey. The main window (displaying the message “The survey is empty. Drag an element from the toolbox or click the button below.” in the picture above) is where survey questions are displayed and edited. On the left is the “toolbox” column, with different question options that you can drag into the main window.
The Preview tab is where you can preview your survey as you design it (see here for more details).
The Logic tab can be used to set up and see survey logics, which I cover in more detail below.
The JSON Editor tab is for advanced editing using the survey’s source code (not covered in this documentation).
All the way to the right is also the Survey settings button (highlighted in the picture), with Undo and Redo arrows to the left of it.
Above that are the survey management buttons:
If you have a wide display (your screen is bigger), you may see a slightly different interface, as the settings panel is open by default on the right (see picture below). As you edit your survey and click on different components of your survey (pages, questions, etc.), the settings panel will update to display the settings for the corresponding component.
Survey designer (wide display version)
There are many question types in Pavlovia surveys, which you can find in the toolbox on the left of the survey designer. These question types are very similar to those found in Qualtrics if you have experience with this platform, although they do not share the same names.
The picture below shows what question type each icon represents.
Question types
The question types available are described below, however I recommend playing around with different types to see what suits your needs best.
To add a new question, click on “Add question” or on any of the question icons in the toolbox. You can also drag the question icon anywhere on the survey to place a new question where you need in the survey.
Each question has the same structure and options (see picture below):
Question structure
As highlighted at the start of this section, the interface for questions will look slightly different if you have a wider display. The settings panel will be open automatically on the right side of the interface, and the settings button present on the question on smaller displays will not be there. The question toolbox will also display more questions and their names as there is more space on the screen.
Question structure (wide display version)
Pavlovia surveys have a specific structure to enable options such as survey logics and branching.
A survey is composed of questions contained in different pages. If you have used Qualtrics in the past, Pavlovia pages are similar to Qualtrics blocks.
Once you add your first question in your survey, new design and structure options will appear (see picture below).
Survey structure
You can now add a survey title and a description (box 1). These will be seen by the participant and should therefore be clear and informative.
You will also see you now have a Page 1 and a Page 2, which can be renamed (box 2). These names will also be visible to participants.
If you click on a page (rather than on a question), you will also see buttons for page options (box 3). These options are similar to question options:
As with questions, the interface looks slightly different on wider displays. The settings button for pages is not available, and instead the settings panel is automatically open on the right of the interface.
Survey structure (wider display version)
To create a new page, click on the Survey settings button, find the Pages tab, then click the + button to add a new page. See the recording below for a demonstration.
Questions within the same page will be displayed to participants
at the same time, unless some questions are part of a logic (detailed here).
Logic can be applied in your survey to control what questions participants see based on their previous answers. It is also most commonly used to enable informed consent by making sure only those who consent are shown the rest of your survey.
Below is an example where seeing the second question is dependent on a participant’s answer to the first question. There is a demonstration video after the instructions.
Say you want to make sure that participants who report liking fruit (question 1, named fruits1) are shown a second question asking them what their favourite fruit is (question 2, named fruits2) - you want to apply a logic so that fruits2 only shows if the fruits1 answer is “Yes”.
To do this, create both questions first. Then in fruits2, click the Settings button and find the Logic tab in the new panel on the right. In the “Visible if” section, click the magic wand button - this will bring up a new window for you to set your logic parameters.
Click the “Select” button to set which question this logic is dependent on (in this example it’s fruits1). Click the “Equals” button if you need to change what state needs to be met for the logic to work. In this example, we want a specific fruits1 answer so Equals is fine.
You will now be able to set what answer participants need to select to move on to the second question. Under the fruits1 question (“Do you like fruits?” here), click the “Select…” button and choose the answer you need (“Yes” here).
Think of it as saying that if the fruits1 answer = “Yes” then fruits2 will be visible to participants, otherwise it will be skipped.
Once you are happy with your settings, click the green “Apply” button and exit the question settings by clicking outside of the panel.
Survey logic can also be more complex, for example if the
display of a question is dependent on multiple answer possibilities in
previous questions.
In the previous example, if we had a third question (named fruits3) asking participants to tell us about their favourite fruit, and we wanted this question to be visible if participants had picked any of the fruits in fruits2, we would have to add conditions to our logic for fruits3.
There is a video after the instructions to demonstrate this process.
First, follow all the previous steps to get the fruits3 logic settings.
Once there, instead of selecting one answer and ending the logic there, we will add multiple conditions. The first condition is created the same as above.
Click the “Select” button to set which question this logic is dependent on (here it’s fruits2). Click the “Equals” button if you need to change what state needs to be met for the logic to work. In this example, we want a specific fruits2 answer so Equals is fine.
You can now select what answer participants need to choose to move on to the third question. Under the fruits2 question (“What are your favourite fruits?” here), click the “Select…” button and choose the answer you need (I have selected the first one, “Apples”).
Now, rather than clicking Apply and finishing there, we will add new conditions, one for each possible answer.
Click the “Add Condition” button below what you have just created. This will display a new condition for you to set up, similarly to the first one you did. However, you will see there is a new “and” button.
The “and” button allows you to control how the conditions work together when applying the logic to your question. For example, if you select “and”, condition 1 and condition 2 need to be met for the logic to be applied. In this example, that means participants need to have responded “Apples” and “Bananas” (for example) in the second question (fruits2) for the third question (fruits3) to be shown to them.
If you click the “and” button, you will see a second option, “or”. When selected, the “or” option means that condition 1 or condition 2 need to be met for the logic to be applied. In this example, that means participants need to have responded “Apples” or “Bananas” (for example) in the second question for the third question to be shown to them.
In our case, we want participants to be shown fruits3 if they have responded to fruits2 with “Apples”, or “Bananas”, or “Strawberries”, or “Raspberries.” So in this case, we will create 3 more conditions and select “or” for each one.
In this case, we can also change the “Equals” button to “Any
of”, as we want to show fruits3 if participants have chosen any of the
answers in fruits2. This is faster than picking each question separately
as above, but does not work in all cases. The video below demonstrates
how to do this.
In both these examples, we have kept the “Equals” button as is.
However, there are multiple options which can be used for different
scenarios.
Clicking on the “Equals” button will show you other options you can select (these depend on the question you have selected, not all will show):
There are multiple randomisation options within a Pavlovia survey. You can randomise questions within a page, or randomise items within certain question types. You can also randomise pages using the new flow view.
If you want to randomise questions within a page, open the page’s settings, navigate to the Layout tab in the settings panel, and select the “Random” box under “Element order on the page” (see picture below).
Please be aware that currently, if you randomise questions within a page and also have logics in place on some of these questions, if logics are met the newly visible question will appear randomly on the page (including potentially before the question the logic was dependent on). For example, if you have 4 questions on a page with randomisation, and question 4 is dependent on question 3, if question 4 is shown it may be shown anywhere on the page, including before question 3.
Randomisation within pages
If you want to randomise items within Checkbox, Radiogroup, Ranking or Image Picker question types, open the question’s settings, navigate to the Choices tab in the settings panel, and select “Random” in the drop-down menu under “Choices sort order” (see picture below).
Randomisation within Checkbox, Radiogroup, Ranking or Image Picker questions
If you want to randomise items within a Single-Choice Matrix question, open the question’s settings, navigate to the Rows tab in the settings panel, and select the “Random” box under “Row order” (see picture below).
Randomisation within Single-Choice Matrix questions
If you have multiple pages/different sets of questions, and you wish for these to be displayed to participants in a random order, you can make use of the new flow view options in Pavlovia surveys.
In Pavlovia, you will see the flow view greyed-out when you first access the Designer tab in your survey, which you can toggle by clicking on the button pictured in the red box below (the button looks like a small grid made of 3x3 black squares).
Greyed-out flow view in Designer tab
Once you’ve toggled the flow view, you will see the flow of your survey and different components you can use. There are 3 components you can use to edit the flow of your survey (see box 1 of the screenshot below):
block component acts like a survey - each block
will have one or more pages and questions. When you create a
block, it will look like a standalone surveysequence component allows you to display a group of
blocks sequentially (one after the other, in the order you put
them)shuffle component allows you to display a group of
blocks randomly (the component will shuffle the blocks and show them in
a random order)With this feature, you can create separate blocks, each with their own pages and questions. You can then insert these blocks in either a sequence or a shuffle component, which will determine the order in which the blocks are shown (see box 2 of the screenshot below).
Flow view components
An example use of this feature is if you have different questionnaires and you wish to show these in a random order to your participants. Each questionnaire would be in a different block, and these blocks would then be inserted in a shuffle component which would display the blocks in a random order.
However, be aware that currently there isn’t a way to ensure an even distribution of participants across different display orders. For example, if you have two questionnaires, you cannot specify that each questionnaire should be seen first by an equal number of participants (e.g. 20 participants see questionnaire 1 first, and 20 see questionnaire 2 first). If you wish to do this you will need to integrate your surveys with an online PsychoPy experiment.
I will follow the example above to demonstrate how to set up a shuffling survey below (you can find the video demonstration after the instructions).
In a new survey, create your first questionnaire as you normally
would. When this is done, toggle the flow view. You will see that you
already have an orange box, most likely named after your survey name,
which is the overall sequence component in which the whole
survey will be contained. Inside, there is one block - this
is your first questionnaire which you have already created.
Now drag and drop a shuffle component into the orange
box, then drag your existing block inside it. Drag a second
block inside the blue box, click on it, then click Add
question to start creating your second questionnaire.
You now have two blocks in your shuffle
component - if you try to preview your survey a few times, you should
see that the first questionnaire to display is random.
You can continue to do this for as many questionnaires as you have, adding a block per questionnaire into the shuffle component.
Once you’re done, add one last block outside the shuffle component but inside the orange sequence component, and make this your debriefing page.
Please remember that you will still need to set up informed
consent and redirection (if the participant does not consent) at the
start of your survey for all participants. Currently there isn’t a way
to set this up in the same survey, so you will need to create an
information and consent survey, which redirects to the shuffling survey
if the participant consents, or ends if they don’t.
To do this, follow the instructions under Getting informed consent, and specifically the section on skipping without completing. You can then add the link to your shuffling survey in the Overview tab, under the Settings section, in the Completion Url box.
You can see what your survey looks like as you are designing it by clicking on the Preview tab (next to the Designer tab, see picture below).
Survey Preview tab
There, you will be able to go through your survey to see how it will be presented to participants. You will also be able to test logics you have put in place.
This is different to piloting your survey, which you can also do to ensure your survey is working the way you want it (this will not enable participants to see the survey, but rather will allow you to test it yourself). More details on your survey’s status (including the Piloting status) can be found here.
Any surveys you are creating to collect data for a study will need the first page to be an information sheet and consent form for participants to provide informed consent to participate.
Additionally, the survey should automatically end if participants choose not to consent.
Below is a video demonstrating the whole process, and written instructions can be found after the video.